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Abstract 

In this , we consider signals with a low-rank covariance matrix which reside in a low-dimensional 
subspace and can be written in terms of a finite (small) number of parameters. Although such signals 
do not necessarily have a sparse representation in a finite basis, they possess a sparse structure which 
makes it possible to recover the signal from compressed measurements. We study the statistical perfor¬ 
mance bound for parameter estimation in the low-rank signal model from compressed measurements. 
Specifically, we derive the Cramer-Rao bound (CRB) for a generic low-rank model and we show that 
the number of compressed samples needs to be larger than the number of sources for the existence of an 
unbiased estimator with finite estimation variance. We further consider the applications to direction-of- 
arrival (DOA) and spectral estimation which fit into the low-rank signal model. We also investigate the 
effect of compression on the CRB by considering numerical examples of the DOA estimation scenario, 
and show how the CRB increases by increasing the compression or equivalently reducing the number 
of compressed samples. 


Index Terms 

Cramer-Rao bound, compressed sensing, low-rank model, DOA estimation, spectral estimation. 

I. Introduction 

Signals with sparse representations can be recovered from much less number of measurements 
than the number of samples given by the Nyquist rate using compressed sensing (CS) methods 
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m-m. Such measurements can be obtained by correlating the signal with a number of sensing 
waveforms m-m. The algorithms used for recovering the signals from such measurements 
exploit the sparsity of the signals in a proper basis (see (31, IfFOl - lfTTl to mention just a few 
existing algorithms). 

There are signals which inherently possess a sparse structure meaning that they can be defined 
by a small number of parameters. However, such signals may not necessarily be represented as 
sparse signals using a proper finite basis, i.e., there may not exist or be known a finite basis 
such that the transformation of the signal to that basis results in a small number of non-zero 
coefficients. For example, consider a signal composed of a linear combination of sinusoids. 
Such a signal generates sparse coefficients by the discrete-time Fourier transform (DTFT), but 
its representation in the Fourier basis obtained by the discrete Fourier transform (DFT) exhibits 
frequency leakage (18) . Although the DTFT is a proper transformation, as it results in a small 
number of non-zero coefficients for the considered signal, it is not a finite basis and cannot be 
used in conventional CS recovery methods which rely on a finite sparsity basis. Such methods 
have poor performance for the considered signals if the DFT basis is used (19) . In this , we 
consider a general class of sparse signals which are represented by a small number of parameters 
in a low-rank signal model. Our goal is to study the performance bounds for the estimation 
of unknown parameters and also the reconstruction of this class of signals from compressed 
measurements. 

The Cramer-Rao bound (CRB) (20) for estimating a sparse parameter vector from compressed 
measurements has been studied in (2T1 . However, the signal model in (2D considers signals which 
can be represented by a finite sparsity basis. Then, the CRB is computed using approaches from 
the theory of constrained CRB in (221 and (23). The constrained CRB for estimating a low- 
rank matrix from compressed measurements has been studied in (24). In this , we consider a 
different signal model which does not involve the constraint on the rank of a matrix. The CRB for 
parameter estimation in compressed sensing has been also studied in (25l - (27) . In (25) . the signal 
of interest is assumed to be a function of real-valued parameters, and it is not assumed to be 
necessarily sparse in a finite basis. The CRB is computed and bounded for different realizations 
of the measurement matrix. The signal model considered in (26) and (27) is different from the 
one studied in this in two aspects. Firstly, in (26) and (271 . a noiseless signal is first compressed 
and then white noise is added to the compressed signal. In contrast, we first add the noise 
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to the signal and then the result is compressed. This results in a different distribution for the 
compressed measurements. Secondly and more significantly, in ll26l and lf27l . the signal is a 
vector which depends on a number of parameters, whereas in this , the signal is composed of a 
parametrized matrix multiplied by a vector of coefficients. This structure of the signal enables 
us to derive a closed-form expression for the CRB of the parameters. 

In this , we extend the results of 11251 for a low-rank signal model. We derive the CRB 
for real and complex-valued parameters. Furthermore, multiple signal snapshots are considered, 
whereas in ll25l . the signal model consists of only a single signal snapshot. We also study the 
minimum number of compressed samples required for unbiased estimation with finite variance. 
Furthermore, the applications to direction-of-arrival (DOA) and spectral estimation which fit into 
the low-rank signal model are also studied. Finally, numerical examples for the DOA estimation 
problem are given to illustrate the effect of compression on the CRB. 

II. Signal Model 

Consider the signal x(t) G C NxXl at time instant t to be of the form 

x(t) = Ad(t) (1) 

where A G C NxXK is a tall matrix (the number of rows is much larger than the number of 
columns), d(t) G C /Cxl is a vector containing unknown amplitudes, and 1 < t < N. A practically 
important example of A is given in Section [V] Since A is a tall matrix, the covariance matrix 
of the signal is a low-rank matrix. Therefore, such a signal is called low-rank. Matrix A can 
be fully known, known up to a number of unknown parameters, or completely unknown. In this 
, we study the second case where matrix A has a known structure, but it contains P number 
of unknown parameters O = [cui, • ■ • ,cop] T G M Px 1 where {■)' stands for the transposition 
operator. 

Let the vector of the measurements y G C 7Vl ' xl be given by 

y(t) = &(x(t) + w(t)) 

= $>x(t) + n{t) (2) 

where G M. NyXNx is the measurement matrix with N y < N x . The additive noise wit) G C Nxx 1 
is assumed to have the circularly-symmetric complex jointly-Gaussian distribution Ar;(0. a 2 /,yj 
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where In x is the identity matrix of size N x and a 2 is the noise power. No specific structure for 
the measurement matrix $ needs to be considered in our derivations. It is because $ is assumed 
to be known at the signal reconstruction stage, and therefore, it is treated as a deterministic 
matrix in our derivations. As a result, irrespective to how <3? is generated, the measurement noise 
n(t) e C NyXl has Gaussian distribution J\fc(0, R) where R = cr 2 $$ i . 


III. Derivation of the CRB 

In this section, we derive the CRB for the signal model given by ([!) and (©. 
First, let the vector of parameters be defined as 


d = 


d (l),d (!),•■■ ,d'(JV),d ( N),n 1 


1 T 


where d(t) and d(t) represent the real and imaginary parts of d(t), respectively. 
The likelihood function of the compressed measurements © is given by 

1 


p(y (!),••• ,y(N)\d) = 


Tl N y N I R 


N 


X 


exp { _ ~ Bd ^)) H R 1 (vit) - Bd(t)) | 

^ t =1 ' 


(3) 


(4) 


where B = <&A and {-) H stands for the Hermitian transposition operator. The log-likelihood 
function can be found by taking the natural logarithm of © as 


LL = lnp (l/(l), • • • ,y(N)\d) 

= -NyN Inn - N In \R\ 

N 

- J2 (' y(t ) - Bd(t)) H R 1 ( y(t) - Bd(t)) . (5) 

t =1 

For brevity, the notation LL will be used in the rest of the to refer to the log-likelihood function 
©. The Fisher information matrix (FIM) is given by 

1(d) = E {^ T } (6) 

where ?/> = dLL/dd. The CRB covariance matrix for the vector of parameters d is then given 
by 

CRB (d) = I~ 1 (d). (7) 
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The derivatives of the LL with respect to d(t) and d(t) are given by 


DLL 
dd(t ) 


= B H R~ 1 n{t) + {n H (t)R- l B\ 
= 2 Re{B H R~ 1 n{t)} 


( 8 ) 


and 


DLL 

dd(t) 


= -]B H R~ 1 n{t) + j (n H (t)R- l B) 
= 2/m {B h R~ l n(t)} 


(9) 


where Re{-} and Im{-} stand for the real part and imaginary part operators, respectively. Recall 
that n(t) = y(t ) — Bd(t) is the measurement noise introduced in ©. 

Note that A has a known structure and contains P unknown parameters cui, ■ • • , c op. Therefore, 
the derivative of the LL with respect to u p for 1 < p < P can be found as 

dLL d H (t)^R~'n(t) + n H {t)R~^d{t) 


duj n 


t =i 
N 


2'£ Re {d H (t)^-^ T R-'n(t)Y 


( 10 ) 


The derivatives of the LL with respect to the whole vector ft, can be then written in matrix form 
as 


DLL 

~dft 


N 


2^Re{D H (t)^ T R- 1 n(t)} 


(ID 


t =i 


where the matrix D(t ) G C NxXP is given by 

m ^ 


dA . dA 

—d(t),--- — d{t) 

u(jJl OUJp 


dA 


dA 


doj\ ' ’ dup 

with <8) standing for the Kronecker product. 


(I P 8) d(t)) 


( 12 ) 
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To proceed, we use the following identities If28l . For two arbitrary complex vectors p and q. 


we have 

Re(p)Re ( q T ) = ^ (Re (pq T ) + Re (pq H )) (13) 

Im(p)Im ( q T ) = ~ (Re (pq T ) - Re (; pq H )) (14) 

Re(p)Im ( q T ) = ^ (im (pq T ) — Im (pq 11 )) . (15) 

Using (fl4l) . (fT5T) . and the fact that for 1 < r, s < iV 

U {n(r)n r (s)} = 0 (16) 

E{n(r)n H (s)} = S rtS R (17) 


where d r „ s denotes the Kronecker delta, we can compute the submatrices of I('d) as 

£ (m) (m) T = ^{e{b»r- W{ s)r-'b}} 

= 2Re{B H R- 1 B) 5r )S (18) 

E {mi) iwsi) = - 2 H E { BHR - ln(r)nH( - s)R - lB }} 

= -2Im {B H R- 1 B} 8 T)S (19) 

rjn y 

E (^)) (W 

= 2Re{B H R- 1 3>D(r)j (20) 

B (S) (m) = 2 H E { B " R ~ In ^ nH ^ R ~ lB }} 

= 2Re{B H R- 1 B) 5 T)S (21) 

j, y 

E (m) © = 

= -2Im^D H (r)^ T R- 1 By 

= 2 Im{B H R- l $D(r)} (22) 

(isr) = 2 E EW^r)}} 

V / V / t =1 r=l 

N 

= 2^Re{D H (t)$ T R- l $D(t)} . (23) 
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Then, 1(d) can be found as 


m = 


H 

11 



Ax 

11 

H 

0 


Ax 

0 


H 

11 

Axv 



11 

11 

Axv 

A[ 

A T 

A, •• 

h fe; 

A T 

r 


(24) 


where (•) and (•) stand for the real and imaginary parts of a matrix, and 


H = 2B H RT 1 B 


A r = 2B H R L $D(r) 

N 

T = 2 Re {D H (t)® T R- l <f>D(t)} . 


(25) 

(26) 

(27) 


t=i 


It is shown in 
for Q is given by 


that for FIM with the structure given in (1241) . the CRB covariance matrix 


N 


CRB' 1 (O) = r - Re { Af H 1 A t } . 


(28) 


t =i 


Using 


, the CRB for can be found in closed-form as 

N 

CRB -1 (O) = 2^Re^D H (t)^ T R~ 1 ^I Ny 

t =i 

-5 ( B H R l B) ~ l $D 


(29) 

Given /” 1 (?7), the covariance matrix of any unbiased estimator of x(t), i.e., C*( t ), satisfies 
the inequality 

- f)cr. H (t) 

(30) 


C* w - > 0. 






The signal x(t) can be written as 

x(t) = Ad(t) = Ad(t) + jAd(t). 

The derivative of x(t) with respect to the vector of unknown parameters d is given by 

dx(t) 


dd 


= [e t <g )[A,jA],D(t)\ 


(31) 


(32) 
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where e t is a row vector of length N with all its elements equal to zero except for the / - 1h 
element which is equal to 1. Finally, by summing over the diagonal elements of (l30l) . we obtain 

(33) 

Similar to (l29l ) . the results in (l32l) and (1331) can also be regarded as closed-form as they can 
be used for analysis, fast computations, and getting insights without requiring Monte-Carlo 
simulations as shown in the next section. It is worth noting that the derived CRB (especially 
(l29l) l can be also used for selecting/optimizing the measurement matrix $ as the CRB depends 
on a specific selection of <F. 

IV. Minimum Number of Compressed Samples 

In this section, we show that if the number of compressed samples is less than or equal to the 
number of sources (N y < K ), the FIM /($) is singular. It is shown in ll29l that a singular FIM 
means that unbiased estimation of the entire parameter vector with finite variance is impossible. 

Let us start with the case that N y < K. In this case, we have rank {B} < K since B e 
C NyxK a s a resu it 5 we also have rank{H} < K (see (l25l) l. and therefore, H is singular. 
Consequently, there exists a nonzero vector u = u + ju G C A xl such that Hu = 0. Therefore, 
( H + jH ] {u + ju) = 0, which can be written in matrix form as 


1 

Hi‘ 

I 

Hi' 

_1 


u 

// H 


u 


Let v = \u T , ii T , Of e RV NK + p )*\ Finally, using (l24l) . we have v T I{d)v = 0, which means 
that I(j&) has a zero eigenvalue, and therefore, it is singular. For the case that N y = K, if 
rank {B} < K, the singularity of the FIM follows from the discussion above. 

Now, consider the case that B is full-rank. Thus, H is invertible. Consider the structure of 
/($) in (l24l) and let the block of all the real and imaginary parts of H be denoted by T. It is 
shown in If28l that for an invertible matrix H , matrix T is also invertible. The Schur complement 
of T denoted by I($)/T is equal to the inverse of the CRB covariance matrix for f2 as given 
in (|29|) . Matrix B is invertible since it is square and full-rank. Therefore, we have 

I Nv -B(B h R- 1 B)~ 1 B h RT 1 = 0. (35) 
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As a result, I{d)/T = 0 (see (l29l». According to the rank additivity formula [[301 . we have 

rank {1(d)} = rank {T} + rank {1(d)/T} = rank {T} . (36) 

Therefore, 1(d) is rank-deficient or equivalently singular. 

Remark. As shown above, if the number of compressed samples is less than or equal to the 
number of sources, the FIM is necessarily singular. However, if the number of compressed 
samples increases, it does not necessarily result in a non-singular FIM for a few more samples. 
Thus, the converse does not hold in general. The minimum number of compressed samples for 
satisfactory performance depends on a specific performance criterion and the estimation method 
used. For example, the minimum number of compressed samples can be chosen to bound the 
probability of a subspace swap BTI or to bound the error of signal subspace estimation Il32l . 
The required number of compressed samples can also be studied from a geometric point of view 

m. 


V. Application Examples 

For the problems of DOA and spectral estimation, d(t ) consists of the amplitudes of K 
number of sources at time instant t. The number of parameters in A is also equal to the number 
of sources, i.e., P = K. Furthermore, A has the structure given by 

A= [a (cui), • • • ,a (u K )} (37) 

where a ( 04 .) for 1 < k < K is the steering vector corresponding to the /;:-th source. Let us 
define c(uj) as the derivative of a(u) with respect to u>, i.e., c(u>) = da(u)/duj. Then, D(t ) 
given by (fl 2 l) can be simplified to 

D(t) = [c (cui) di(t), ■ ■ ■ , c (c o K ) d K {t)\ 

= [c (wi), • • • , c (u K )\ diag (d(t)} (38) 

where d k (t ) is the k- th element of d(t) and the diag {•} operator converts a vector into a diagonal 
matrix. 
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VI. Numerical Results 

In this section, the application of the derived CRB formulas for the problem of DOA estimation 
is illustrated. Our goal is to investigate the performance bounds for unbiased estimators when 
the signal is compressed at different rates. 

Consider K = 11 equally spaced sources impinging on a uniform linear array of N x = 50 
antenna elements from directions u)\ = 20° x (7r/180), CU 2 = 23° x (7t/180), • • • , u\\ = 50° x 
(7t/ 180). The steering vector of the array a(u) is given by 

A ^ e -i 27 r(d/A)sin(o;)^ _ . . ^ g-j 27 r(JV a: -l)(d/A) sin(a)) j T (39) 

where d is the interelement spacing of the array and A is the wavelength of the plane wave 
impinging on the array. In our numerical example, d/X is set to 0.5. The number of snapshots 
is also set to Af = 10. Each source vector d(t) is considered to be independent from the 
source vectors at other time instances and is drawn from the circularly-symmetric complex 
jointly-Gaussian distribution J\f c (0, <j 2 Ik)- The signal-to-noise ratio (SNR) is set to SNR = 
101og 10 (a 2 /a 2 ) = 10 dB. The source vectors are drawn once and kept unchanged. 

Fig. |Tj shows the CRB for estimating luq = 35 ° x (7r/180) versus the number of compressed 
samples N y . For the case when N y — N x — 50, the measurement matrix $ is set to the identity 
matrix. Then, $ is initialized for N y = 49 by drawing samples from the Gaussian distribution 
jV(0,1/49). For the rest of N y values, the first N y rows of the initial matrix $ are scaled by 
y/AQ/Ny and used to obtain the CRB. 

As expected, it can be seen in Fig. |T| that the CRB increases as the number of compressed 
samples N y reduces. The minimum number of compressed samples is set to N y = 12 which 
is equal to the number of sources plus one (K + 1). As shown in Section [TV] if the number 
of compressed samples is equal to or less than the number of sources, there can be no unbi¬ 
ased estimator with a finite estimation variance. Otherwise, if the CRB exists, there also exist 
estimators lfT8l that achieve it. 
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Number of compressed samples (N y ) 


Fig. 1. CRB for estimating 0 J 6 = 35 ° x (7r/180). 


VII. Conclusion 

The class of signals fitting a low-rank signal model has been considered in this . Such signals 
are inherently sparse according to the signal model and can be recovered from compressed 
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measurements. We have studied the performance bounds for unbiased estimators of parameters of 
such low-rank signal model from compressed samples. The Cramer-Rao bound has been derived 
for a generic low-rank model and it has been shown that the number of compressed samples 
needs to be at least larger than the number of sources for the existence of an unbiased estimator 
with finite variance. Furthermore, the applications to DOA and spectral estimation have been 
considered. Numerical examples have been also given to illustrate the effect of compression 
on the CRB. It has been shown how the CRB increases until the point where the number of 
compressed samples is larger than the number of sources. For lower number of compressed 
samples, the CRB becomes unbounded. 
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